[Profiler] Allow tests for .NET Framework #5948

chrisnas · 2024-08-27T08:20:18Z

Summary of changes

Record and replay ETW events received by the Datadog agent.

Reason for change

Since ETW events sent by the .NET Framework are proxied by the Datadog agent, it is needed to mock this data exchange in integration tests.

Implementation details

events are recorded by the existing tooling to easily build serialized events files
events are replayed by a new C# tool using classes that will be used by the integration tests

Test coverage

manual for the tooling
new integration tests for .NET framework

Other details

andrewlock · 2024-08-27T08:55:05Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5948) - mean (70ms)  : 68, 72
     .   : milestone, 70,
    master - mean (70ms)  : 67, 73
     .   : milestone, 70,

    section CallTarget+Inlining+NGEN
    This PR (5948) - mean (1,116ms)  : 1088, 1145
     .   : milestone, 1116,
    master - mean (1,111ms)  : 1087, 1134
     .   : milestone, 1111,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5948) - mean (109ms)  : 106, 111
     .   : milestone, 109,
    master - mean (109ms)  : 105, 113
     .   : milestone, 109,

    section CallTarget+Inlining+NGEN
    This PR (5948) - mean (795ms)  : 672, 919
     .   : milestone, 795,
    master - mean (770ms)  : 756, 784
     .   : milestone, 770,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5948) - mean (92ms)  : 90, 94
     .   : milestone, 92,
    master - mean (92ms)  : 90, 94
     .   : milestone, 92,

    section CallTarget+Inlining+NGEN
    This PR (5948) - mean (725ms)  : 707, 743
     .   : milestone, 725,
    master - mean (727ms)  : 710, 744
     .   : milestone, 727,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5948) - mean (190ms)  : 186, 194
     .   : milestone, 190,
    master - mean (190ms)  : 187, 193
     .   : milestone, 190,

    section CallTarget+Inlining+NGEN
    This PR (5948) - mean (1,205ms)  : 1182, 1229
     .   : milestone, 1205,
    master - mean (1,199ms)  : 1173, 1225
     .   : milestone, 1199,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5948) - mean (275ms)  : 270, 279
     .   : milestone, 275,
    master - mean (275ms)  : 270, 280
     .   : milestone, 275,

    section CallTarget+Inlining+NGEN
    This PR (5948) - mean (945ms)  : 922, 968
     .   : milestone, 945,
    master - mean (945ms)  : 928, 963
     .   : milestone, 945,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5948) - mean (265ms)  : 260, 269
     .   : milestone, 265,
    master - mean (264ms)  : 259, 268
     .   : milestone, 264,

    section CallTarget+Inlining+NGEN
    This PR (5948) - mean (926ms)  : 910, 943
     .   : milestone, 926,
    master - mean (922ms)  : 899, 946
     .   : milestone, 922,

datadog-ddstaging · 2024-08-27T09:03:43Z

Datadog Report

Branch report: chrisnas/tests_for_framework
Commit report: c6022d1
Test service: dd-trace-dotnet

❌ 5 Failed (0 Known Flaky), 164683 Passed, 1270 Skipped, 11h 39m 54.76s Total Time

❌ Failed Tests (5)

ShouldGetAllocationSamplesViaEtw - Datadog.Profiler.IntegrationTests.Allocations.AllocationsProfilerTest - Details
Expand for error
```
 The test Samples.Computer01 is running for too long or was lost
```
ShouldGetAllocationSamplesViaEtw - Datadog.Profiler.IntegrationTests.Allocations.AllocationsProfilerTest - Details
Expand for error
```
 The test Samples.Computer01 is running for too long or was lost
```
ShouldGetLockContentionSamplesViaEtw - Datadog.Profiler.IntegrationTests.Contention.ContentionProfilerTest - Details
Expand for error
```
 The test Samples.Computer01 is running for too long or was lost
```
ShouldGetLockContentionSamplesViaEtw - Datadog.Profiler.IntegrationTests.Contention.ContentionProfilerTest - Details
Expand for error
```
 The test Samples.Computer01 is running for too long or was lost
```
ShouldGetGarbageCollectionSamplesViaEtw - Datadog.Profiler.IntegrationTests.GarbageCollections.GarbageCollectionsProfilerTest - Details
Expand for error
```
 
 Actual:   Falseilure
```

andrewlock · 2024-08-27T09:34:45Z

Benchmarks Report for tracer 🐌

Benchmarks for #5948 compared to master:

1 benchmarks are slower, with geometric mean 1.163
1 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`StartStopWithChild`	net6.0	7.83μs	45ns	334ns	0.0174	0.0087	0	5.42 KB
master	`StartStopWithChild`	netcoreapp3.1	10.2μs	55.6ns	324ns	0.015	0.00501	0	5.62 KB
master	`StartStopWithChild`	net472	16.4μs	47.8ns	185ns	1.03	0.318	0.106	6.07 KB
#5948	`StartStopWithChild`	net6.0	7.82μs	43.9ns	304ns	0.0152	0.00761	0	5.43 KB
#5948	`StartStopWithChild`	netcoreapp3.1	10μs	56ns	363ns	0.0142	0.00473	0	5.61 KB
#5948	`StartStopWithChild`	net472	16.9μs	62.3ns	241ns	1.01	0.297	0.0824	6.06 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	468μs	278ns	1.04μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	643μs	306ns	1.18μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	net472	845μs	630ns	2.44μs	0.419	3.3 KB
#5948	`WriteAndFlushEnrichedTraces`	net6.0	488μs	578ns	2.16μs	0	2.7 KB
#5948	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	640μs	329ns	1.27μs	0	2.7 KB
#5948	`WriteAndFlushEnrichedTraces`	net472	848μs	451ns	1.74μs	0.422	3.3 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendRequest`	net6.0	199μs	1.16μs	11μs	0.193	18.45 KB
master	`SendRequest`	netcoreapp3.1	223μs	1.3μs	12μs	0.214	20.61 KB
master	`SendRequest`	net472	0ns	0ns	0ns	0	0 b
#5948	`SendRequest`	net6.0	195μs	1.12μs	8.63μs	0.191	18.45 KB
#5948	`SendRequest`	netcoreapp3.1	215μs	1.02μs	4.91μs	0.206	20.61 KB
#5948	`SendRequest`	net472	3.54E‑05ns	3.54E‑05ns	0.000128ns	0	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #5948

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net6.0	41.45 KB	41.84 KB	391 B	0.94%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	560μs	2.74μs	12.6μs	0.556	0	0	41.45 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	679μs	3.72μs	21.7μs	0.336	0	0	41.65 KB
master	`WriteAndFlushEnrichedTraces`	net472	892μs	3.6μs	14μs	8.09	2.25	0.45	53.29 KB
#5948	`WriteAndFlushEnrichedTraces`	net6.0	580μs	2.68μs	10.4μs	0.558	0	0	41.84 KB
#5948	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	684μs	2.07μs	8.01μs	0.336	0	0	41.74 KB
#5948	`WriteAndFlushEnrichedTraces`	net472	857μs	3.57μs	13.4μs	8.36	2.64	0.44	53.33 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteNonQuery`	net6.0	1.26μs	0.998ns	3.87ns	0.0146	1.02 KB
master	`ExecuteNonQuery`	netcoreapp3.1	1.78μs	0.923ns	3.57ns	0.0131	1.02 KB
master	`ExecuteNonQuery`	net472	2.11μs	2.27ns	8.78ns	0.156	987 B
#5948	`ExecuteNonQuery`	net6.0	1.23μs	0.892ns	3.34ns	0.0148	1.02 KB
#5948	`ExecuteNonQuery`	netcoreapp3.1	1.76μs	1.25ns	4.83ns	0.0132	1.02 KB
#5948	`ExecuteNonQuery`	net472	2.06μs	2.14ns	8.31ns	0.156	987 B

Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`CallElasticsearch`	net6.0	1.16μs	1.26ns	4.9ns	0.0133	976 B
master	`CallElasticsearch`	netcoreapp3.1	1.52μs	0.803ns	3.11ns	0.0133	976 B
master	`CallElasticsearch`	net472	2.47μs	2.06ns	7.96ns	0.158	995 B
master	`CallElasticsearchAsync`	net6.0	1.34μs	0.857ns	3.32ns	0.0134	952 B
master	`CallElasticsearchAsync`	netcoreapp3.1	1.63μs	0.472ns	1.77ns	0.0141	1.02 KB
master	`CallElasticsearchAsync`	net472	2.73μs	1.57ns	6.09ns	0.166	1.05 KB
#5948	`CallElasticsearch`	net6.0	1.2μs	0.852ns	3.3ns	0.0133	976 B
#5948	`CallElasticsearch`	netcoreapp3.1	1.57μs	0.911ns	3.41ns	0.0126	976 B
#5948	`CallElasticsearch`	net472	2.49μs	1.73ns	6.48ns	0.158	995 B
#5948	`CallElasticsearchAsync`	net6.0	1.26μs	0.595ns	2.3ns	0.0132	952 B
#5948	`CallElasticsearchAsync`	netcoreapp3.1	1.59μs	2.26ns	8.47ns	0.0141	1.02 KB
#5948	`CallElasticsearchAsync`	net472	2.6μs	1.58ns	5.69ns	0.167	1.05 KB

Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteAsync`	net6.0	1.27μs	1.51ns	5.86ns	0.0131	952 B
master	`ExecuteAsync`	netcoreapp3.1	1.61μs	0.606ns	2.35ns	0.0129	952 B
master	`ExecuteAsync`	net472	1.73μs	0.91ns	3.41ns	0.145	915 B
#5948	`ExecuteAsync`	net6.0	1.36μs	1.18ns	4.56ns	0.0129	952 B
#5948	`ExecuteAsync`	netcoreapp3.1	1.55μs	0.889ns	3.08ns	0.0127	952 B
#5948	`ExecuteAsync`	net472	1.76μs	2.13ns	8.23ns	0.145	915 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net6.0	4.16μs	1.24ns	4.65ns	0.0311	2.22 KB
master	`SendAsync`	netcoreapp3.1	5.11μs	2.7ns	10.5ns	0.0359	2.76 KB
master	`SendAsync`	net472	7.89μs	2.34ns	9.08ns	0.497	3.15 KB
#5948	`SendAsync`	net6.0	4.16μs	1.42ns	5.32ns	0.0311	2.22 KB
#5948	`SendAsync`	netcoreapp3.1	5.18μs	1.76ns	6.82ns	0.0388	2.76 KB
#5948	`SendAsync`	net472	7.86μs	2.84ns	11ns	0.5	3.15 KB

Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	1.46μs	1.4ns	5.61ns	0.0232	1.64 KB
master	`EnrichedLog`	netcoreapp3.1	2.34μs	1.91ns	7.38ns	0.0221	1.64 KB
master	`EnrichedLog`	net472	2.7μs	1.56ns	6.05ns	0.249	1.57 KB
#5948	`EnrichedLog`	net6.0	1.52μs	1.26ns	4.86ns	0.0228	1.64 KB
#5948	`EnrichedLog`	netcoreapp3.1	2.19μs	1.38ns	5.35ns	0.0218	1.64 KB
#5948	`EnrichedLog`	net472	2.54μs	1.81ns	6.76ns	0.249	1.57 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EnrichedLog`	net6.0	118μs	215ns	834ns	0	0	4.28 KB
master	`EnrichedLog`	netcoreapp3.1	122μs	184ns	712ns	0	0	4.28 KB
master	`EnrichedLog`	net472	153μs	127ns	491ns	0.686	0.229	4.46 KB
#5948	`EnrichedLog`	net6.0	116μs	142ns	549ns	0	0	4.28 KB
#5948	`EnrichedLog`	netcoreapp3.1	122μs	146ns	566ns	0	0	4.28 KB
#5948	`EnrichedLog`	net472	152μs	231ns	894ns	0.681	0.227	4.46 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.95μs	1.03ns	3.71ns	0.031	2.2 KB
master	`EnrichedLog`	netcoreapp3.1	4.15μs	1.69ns	6.54ns	0.0291	2.2 KB
master	`EnrichedLog`	net472	4.72μs	1.17ns	4.53ns	0.32	2.02 KB
#5948	`EnrichedLog`	net6.0	3.02μs	1.26ns	4.86ns	0.0304	2.2 KB
#5948	`EnrichedLog`	netcoreapp3.1	4.14μs	2.46ns	9.54ns	0.0292	2.2 KB
#5948	`EnrichedLog`	net472	5.01μs	3.66ns	14.2ns	0.319	2.02 KB

Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`SendReceive`	net6.0	1.42μs	0.882ns	3.3ns	0.0156	0	1.14 KB
master	`SendReceive`	netcoreapp3.1	1.69μs	1.03ns	3.57ns	0.0153	0	1.14 KB
master	`SendReceive`	net472	2.19μs	1.56ns	6.03ns	0.183	0.00109	1.16 KB
#5948	`SendReceive`	net6.0	1.44μs	1.1ns	4.26ns	0.0158	0	1.14 KB
#5948	`SendReceive`	netcoreapp3.1	1.8μs	0.981ns	3.8ns	0.0153	0	1.14 KB
#5948	`SendReceive`	net472	2.13μs	0.896ns	3.23ns	0.183	0.00107	1.16 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.73μs	1.15ns	4.15ns	0.0221	1.6 KB
master	`EnrichedLog`	netcoreapp3.1	3.88μs	1.74ns	6.49ns	0.0214	1.65 KB
master	`EnrichedLog`	net472	4.28μs	2.92ns	11.3ns	0.323	2.04 KB
#5948	`EnrichedLog`	net6.0	2.72μs	0.453ns	1.57ns	0.022	1.6 KB
#5948	`EnrichedLog`	netcoreapp3.1	3.92μs	1.57ns	6.06ns	0.0215	1.65 KB
#5948	`EnrichedLog`	net472	4.28μs	3.81ns	13.7ns	0.323	2.04 KB

Benchmarks.Trace.SpanBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #5948

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net6.0	1.163	394.48	458.83

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net6.0	394ns	0.178ns	0.667ns	0.00815	576 B
master	`StartFinishSpan`	netcoreapp3.1	552ns	0.411ns	1.48ns	0.00778	576 B
master	`StartFinishSpan`	net472	774ns	0.569ns	2.2ns	0.0915	578 B
master	`StartFinishScope`	net6.0	481ns	0.221ns	0.825ns	0.00986	696 B
master	`StartFinishScope`	netcoreapp3.1	733ns	1.4ns	5.43ns	0.00935	696 B
master	`StartFinishScope`	net472	947ns	0.751ns	2.91ns	0.104	658 B
#5948	`StartFinishSpan`	net6.0	459ns	0.178ns	0.691ns	0.00806	576 B
#5948	`StartFinishSpan`	netcoreapp3.1	615ns	0.539ns	2.09ns	0.00775	576 B
#5948	`StartFinishSpan`	net472	746ns	0.854ns	3.31ns	0.0917	578 B
#5948	`StartFinishScope`	net6.0	487ns	0.34ns	1.32ns	0.00982	696 B
#5948	`StartFinishScope`	netcoreapp3.1	780ns	0.556ns	2.08ns	0.00935	696 B
#5948	`StartFinishScope`	net472	939ns	0.669ns	2.59ns	0.104	658 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunOnMethodBegin`	net6.0	662ns	0.373ns	1.44ns	0.00994	696 B
master	`RunOnMethodBegin`	netcoreapp3.1	927ns	0.475ns	1.84ns	0.00924	696 B
master	`RunOnMethodBegin`	net472	1.13μs	0.284ns	0.984ns	0.104	658 B
#5948	`RunOnMethodBegin`	net6.0	680ns	0.309ns	1.2ns	0.0099	696 B
#5948	`RunOnMethodBegin`	netcoreapp3.1	929ns	0.412ns	1.49ns	0.00935	696 B
#5948	`RunOnMethodBegin`	net472	1.18μs	0.741ns	2.77ns	0.105	658 B

andrewlock · 2024-08-27T20:15:47Z

Throughput/Crank Report ⚡

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5948) (11.197M)   : 0, 11196606
    master (11.189M)   : 0, 11189141
    benchmarks/2.9.0 (11.081M)   : 0, 11080577

    section Automatic
    This PR (5948) (7.387M)   : 0, 7386631
    master (7.207M)   : 0, 7206782
    benchmarks/2.9.0 (7.732M)   : 0, 7732233

    section Trace stats
    master (7.808M)   : 0, 7807510

    section Manual
    master (11.043M)   : 0, 11042576

    section Manual + Automatic
    This PR (5948) (6.801M)   : 0, 6800876
    master (6.772M)   : 0, 6772285

    section DD_TRACE_ENABLED=0
    master (10.192M)   : 0, 10191757

gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5948) (9.574M)   : 0, 9574295
    master (9.484M)   : 0, 9483809
    benchmarks/2.9.0 (9.798M)   : 0, 9798067

    section Automatic
    This PR (5948) (6.507M)   : 0, 6506935
    master (6.638M)   : 0, 6638030

    section Trace stats
    master (6.841M)   : 0, 6841295

    section Manual
    master (9.635M)   : 0, 9634939

    section Manual + Automatic
    This PR (5948) (6.061M)   : 0, 6060844
    master (6.043M)   : 0, 6043070

    section DD_TRACE_ENABLED=0
    master (9.041M)   : 0, 9041061

gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5948) (10.415M)   : 0, 10414652
    master (10.256M)   : 0, 10256202
    benchmarks/2.9.0 (10.067M)   : 0, 10067315

    section Automatic
    This PR (5948) (6.716M)   : 0, 6716075
    master (6.786M)   : 0, 6786136
    benchmarks/2.9.0 (7.552M)   : 0, 7552193

    section Trace stats
    master (7.450M)   : 0, 7449846

    section Manual
    master (10.418M)   : 0, 10418289

    section Manual + Automatic
    This PR (5948) (6.398M)   : 0, 6398217
    master (6.208M)   : 0, 6208168

    section DD_TRACE_ENABLED=0
    master (9.534M)   : 0, 9534292

profiler/src/ProfilerEngine/Datadog.Profiler.Native.Windows/ETW/EtwEventsHandler.cpp

profiler/src/ProfilerEngine/Datadog.Profiler.Native.Windows/EtwEventsManager.cpp

profiler/src/Tools/ETW/dd-prof-etw/dd-prof-etw-client/EtwEventDumper.h

gleocadie

LGTM with few comments

gleocadie · 2024-10-14T11:32:38Z

profiler/src/ProfilerEngine/Datadog.Profiler.Native.Windows/EtwEventsManager.h

@@ -110,6 +110,7 @@ class EtwEventsManager :

 // responsible for receiving ETW events from the Windows Agent
 std::unique_ptr<EtwEventsHandler> _eventsHandler;
+ std::string _agentEndpoint;


_agentEndpoint is for the replay scenario only ?
if yes, maybe make it explicit can help understand that's not a normal case (in prod)

gleocadie · 2024-10-14T11:34:59Z

profiler/src/ProfilerEngine/Datadog.Profiler.Native/ClrEventsParser.cpp

+ std::stringstream buffer;
+ buffer << "OnGarbageCollectionStart: " << number << " " << generation << " " << reason << " " << type;


[PERF] I expect this log to be written in Debug only, so I would expect the std::stringstream to be construction only if we are debug level no?

gleocadie · 2024-10-14T11:35:14Z

profiler/src/ProfilerEngine/Datadog.Profiler.Native/ClrEventsParser.cpp

+ std::stringstream buffer;
+ buffer << "OnGarbageCollectionEnd: " << number << " " << generation << " " << reason << " " << type;
+ LOG_GC_EVENT(buffer.str());


gleocadie · 2024-10-14T11:36:08Z

profiler/src/ProfilerEngine/Datadog.Profiler.Native/Configuration.cpp

@@ -93,6 +93,7 @@ Configuration::Configuration()
 _isEtwEnabled = GetEnvironmentValue(EnvironmentVariables::EtwEnabled, false);
 _deploymentMode = GetEnvironmentValue(EnvironmentVariables::SsiDeployed, DeploymentMode::Manual);
 _isEtwLoggingEnabled = GetEnvironmentValue(EnvironmentVariables::EtwLoggingEnabled, false);
+ _etwEndpoint = GetEnvironmentValue(EnvironmentVariables::EtwEndpoint, DefaultEmptyString);


is the _etwEndpoint only for the replay scenario ? if yes maybe make it clear :)

gleocadie · 2024-10-14T11:36:17Z

profiler/src/ProfilerEngine/Datadog.Profiler.Native/Configuration.cpp

@@ -565,6 +566,15 @@ bool Configuration::IsEtwLoggingEnabled() const
 #endif
 }

+std::string const& Configuration::GetEtwEndpoint() const


same for the method name

gleocadie · 2024-10-14T11:36:34Z

profiler/src/ProfilerEngine/Datadog.Profiler.Native/EnvironmentVariables.h

@@ -63,6 +63,7 @@ class EnvironmentVariables final
 inline static const shared::WSTRING EtwEnabled = WStr("DD_INTERNAL_PROFILING_ETW_ENABLED");
 inline static const shared::WSTRING SsiDeployed = WStr("DD_INJECTION_ENABLED");
 inline static const shared::WSTRING EtwLoggingEnabled = WStr("DD_INTERNAL_ETW_LOGGING_ENABLED");
+ inline static const shared::WSTRING EtwEndpoint = WStr("DD_INTERNAL_ETW_ENDPOINT");


same for the environment variable

chrisnas added 4 commits August 22, 2024 10:07

Start implementation

7f09b45

Merge remote-tracking branch 'origin' into chrisnas/tests_for_framework

a1ceb45

Merge remote-tracking branch 'origin' into chrisnas/tests_for_framework

344bff2

Merge remote-tracking branch 'origin' into chrisnas/tests_for_framework

19394c2

chrisnas requested review from a team as code owners August 27, 2024 08:20

github-actions bot added the area:profiler label Aug 27, 2024

Disable verbose traces

bfe640b

chrisnas added 4 commits August 27, 2024 14:26

Implement replay tool

5b4a91c

Merge remote-tracking branch 'origin' into chrisnas/tests_for_framework

95c2e41

Remove unneeded uses

5fee35d

Remove unneeded uses

9d64afd

chrisnas added 8 commits August 28, 2024 12:03

Support custom namedpipe end point for integration test

6ef8615

Merge remote-tracking branch 'origin' into chrisnas/tests_for_framework

e2b8204

Update recording/replay for whole ETW record

27a1e99

Merge remote-tracking branch 'origin' into chrisnas/tests_for_framework

40713a6

Agent proxy replays ETW events

1192890

Merge remote-tracking branch 'origin' into chrisnas/tests_for_framework

3ba6ec4

Create a proxy agent tool

f8beb93

Fix solution

ef87aad

gleocadie reviewed Sep 3, 2024

View reviewed changes

chrisnas added 5 commits September 4, 2024 09:16

Fix AgentEtwProxy

908223c

Merge remote-tracking branch 'origin' into chrisnas/tests_for_framework

0bae178

Merge remote-tracking branch 'origin' into chrisnas/tests_for_framework

1443430

Add more logging

e2227a5

Add integration tests for GCs and Lock contention (missing stacks)

a3a05eb

chrisnas added 28 commits September 17, 2024 11:44

Fix flacky tests

a481a77

Merge remote-tracking branch 'origin' into chrisnas/tests_for_framework

23daa6f

Fix flaky tests

0c09d05

Fix flacky test

7502ae0

Merge remote-tracking branch 'origin' into chrisnas/tests_for_framework

2a95956

Add logs

81bed76

Merge remote-tracking branch 'origin' into chrisnas/tests_for_framework

17c959a

Allow the generation of a memory dump on Windows for integration tests

ef69bdb

Merge remote-tracking branch 'origin' into chrisnas/tests_for_framework

3338eb8

fix CI

2995e06

Merge remote-tracking branch 'origin' into chrisnas/tests_for_framework

4a28990

Avoid hanging named pipe tests in the CI

c7cb293

Merge remote-tracking branch 'origin' into chrisnas/tests_for_framework

f8cb32f

Merge remote-tracking branch 'origin' into chrisnas/tests_for_framework

9320b68

Merge remote-tracking branch 'origin' into chrisnas/tests_for_framework

713914a

Enable automatic memory dump collection on hang for Windows

84a29d3

Try to avoid hangs when reading/writing to named pipes

2220219

Try to fix CI hang

7824cf5

Merge remote-tracking branch 'origin' into chrisnas/tests_for_framework

9f4e200

Send xunit logs into a file downloadable from CI

59deb38

Merge remote-tracking branch 'origin' into chrisnas/tests_for_framework

c0fb91d

Fix issue in CI

5a03425

Merge remote-tracking branch 'origin' into chrisnas/tests_for_framework

308ad32

Merge remote-tracking branch 'origin' into chrisnas/tests_for_framework

64f6bc9

Cleanup

01f2d2a

Merge remote-tracking branch 'origin' into chrisnas/tests_for_framework

8cd6e1b

Merge remote-tracking branch 'origin' into chrisnas/tests_for_framework

fc22423

Merge remote-tracking branch 'origin' into chrisnas/tests_for_framework

47b37dc

gleocadie approved these changes Oct 14, 2024

View reviewed changes

Take PR feedbacks into account

bb86f42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Profiler] Allow tests for .NET Framework #5948

[Profiler] Allow tests for .NET Framework #5948

chrisnas commented Aug 27, 2024

andrewlock commented Aug 27, 2024 •

edited

Loading

datadog-ddstaging bot commented Aug 27, 2024 •

edited

Loading

andrewlock commented Aug 27, 2024 •

edited

Loading

Raw results

Raw results

Raw results

More allocations ⚠️ in #5948

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #5948

Raw results

Raw results

andrewlock commented Aug 27, 2024 •

edited

Loading

gleocadie left a comment

gleocadie Oct 14, 2024

gleocadie Oct 14, 2024

gleocadie Oct 14, 2024

gleocadie Oct 14, 2024

gleocadie Oct 14, 2024

gleocadie Oct 14, 2024

		std::stringstream buffer;
		buffer << "OnGarbageCollectionStart: " << number << " " << generation << " " << reason << " " << type;

[Profiler] Allow tests for .NET Framework #5948

Are you sure you want to change the base?

[Profiler] Allow tests for .NET Framework #5948

Conversation

chrisnas commented Aug 27, 2024

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

andrewlock commented Aug 27, 2024 • edited Loading

Execution-Time Benchmarks Report ⏱️

datadog-ddstaging bot commented Aug 27, 2024 • edited Loading

Datadog Report

❌ Failed Tests (5)

andrewlock commented Aug 27, 2024 • edited Loading

Benchmarks Report for tracer 🐌

Benchmark details

Raw results

Raw results

Raw results

More allocations ⚠️ in #5948

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #5948

Raw results

Raw results

andrewlock commented Aug 27, 2024 • edited Loading

Throughput/Crank Report ⚡

gleocadie left a comment

Choose a reason for hiding this comment

gleocadie Oct 14, 2024

Choose a reason for hiding this comment

gleocadie Oct 14, 2024

Choose a reason for hiding this comment

gleocadie Oct 14, 2024

Choose a reason for hiding this comment

gleocadie Oct 14, 2024

Choose a reason for hiding this comment

gleocadie Oct 14, 2024

Choose a reason for hiding this comment

gleocadie Oct 14, 2024

Choose a reason for hiding this comment

andrewlock commented Aug 27, 2024 •

edited

Loading

datadog-ddstaging bot commented Aug 27, 2024 •

edited

Loading

andrewlock commented Aug 27, 2024 •

edited

Loading

andrewlock commented Aug 27, 2024 •

edited

Loading